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Training  to  Reduce  the  Use  of  Irrelevant  Information 
in  Personnel  Selection 
by 

Gary  J.  Gaeth  and  James  Shanteau 
INTRODUCTION 

The  fact  that  irrelevant  information  often  has  an 
adverse  effect  on  human  judgment  is  well  established.  For 
example,  researchers  have  previously  shown  that  irrelevant 
information  slows  down  judgment  speed  (Krueger,  1973) , 
increases  judgmental  errors  (Hoyer,  Rebok  &  Sved,  1979),  and 
causes  a  decrement  in  evaluating  relevant  information 
(Troutman,  &  Shanteau,  1977)  .  These  and  similar  studies 
provide  a  first  step  in  understanding  the  role  of 
irrelevance  in  judgment  (Gaeth  &  Shanteau,  1981).  However, 
little  attention  has  been  focused  on  the  next  logical  step: 
the  development  of  training  procedures  to  reduce  the 
adverse  influence  of  irrelevant  information. 

An  initial  effort  in  this  direction  was  taken  by 
Shanteau  and  Gaeth  (1981) .  In  this  earlier  study,  the  main 
concern  was  to  develop  a  training  technique  which  could  be 
used  to  reduce  the  influence  of  irrelevance.  It  was  first 
confirmed  that  irrelevant  materials  adversely  influenced 
agricultural  judges  experienced  in  soil  science.  Then,  two 
training  procedures  were  evaluated;  a  lecture-based  (verbal 
instructions)  training  procedure,  and  an  interactive-based 
(perceptual)  training  procedure. 
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The  results  showed  that  a  combination  of  these  two 
training  procedures  considerably  reduced  the  influence  of 
the  irrelevant  information.  However,  the  lecture  training 
alone  had  little  effect.  A  notable  finding  was  that  while 
no  accuracy  feedback  was  given,  an  increase  in  accuracy  was 
observed  following  training.  That  is,  accuracy  increased  as 
the  use  of  irrelevant  information  decreased. 

It  was  suspected  that  the  interactive  training  was 
largely  responsible  for  both  the  reduction  in  the  influence 
of  irrelevance  and  the  increase  in  accuracy.  Unfortunately, 
because  only  a  small  pool  of  experienced  soil  judges  was 
available  at  KSU  it  was  necessary  to  present  both  the 
lecture  training  and  the  interactive  training  to  every 
subject.  Thus,  it  was  impossible  to  separate  the  effect  of 
the  interactive  training  from  that  of  the  lecture  training. 

In  summary,  our  initial  study  (Shanteau  &  Gaeth,  1981) 
represented  a  major  step  toward  understanding  the  role  of 
irrelevant  information  in  human  judgment.  First,  we 
demonstrated  that  irrelevant  information  is  indeed  a 
problem,  even  for  experienced  judges.  Then,  training 
procedures  were  developed  and  tested  which  improved 
judgments  when  irrelevance  was  present.  Despite  this 
earlier  success,  two  questions  deserve  immediate 
investigation:  (1)  What  is  the  separate  impact  of  lecture 
and  interactive  training?  (2)  Will  the  original  soil 
training  procedures  generalize  to  other  applied  areas?  The 
answers  to  these  two  questions  will  considerably  advance  our 
understanding  of  the  effects  of  irrelevance  and  help  in  the 
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development  of  general  training  techniques  to  deal  with  it. 


A  study  by  Nagy  (1981)  provides  a  recent  demonstration 
of  the  influence  of  irrelevance  in  the  area  of  personnel 
selection.  Initially,  Nagy  was  interested  in  the  influence 
of  biases  on  hiring  judgments.  As  she  points  out,  the 
majority  of  past  research  dealing  with  personnel  selection 
has  focused  on  job-relevant  dimensions  of  information  such 
as  academic  record  (Hakel,  Ohnesorge  &  Dunnette,  1970) , 
background  ienformation  (Mayfield  &  Carlson,  1966) ,  and 
work  experince  (Norman,  1976) .  Recently,  however,  interest 
has  included  investigations  of  the  role  of  job-irrelevant 
information  such  as  sex  and  age. 

After  an  extensive  review  of  the  literature  (the  reader 
is  referred  to  Nagy,  1981) ,  Nagy  selected  the  factors  of 
sex,  age,  and  physical  attractiveness  as  the  three  most 
common  pieces  of  job-irrelevant  information.  They  were  also 
thought  to  be  the  most  likely  to  have  an  influence  in 
personnel  selection.  Despite  previous  literature  indicating 
that  these  job-irrelevant  factors  may  have  a  qualitative 
influence,  the  exact  form  of  this  influence  had  not  been 
studied  quantitatively  prior  to  Nagy's  work. 

After  selecting  these  job-irrelevant  dimensions,  Nagy 
created  a  set  of  hypothetical  job  applicants.  With  this 
set,  she  studied  the  influence  of  job-relevant  (experience 
and  recommendations)  and  job-irrelevant  information  (sex, 
age,  attractiveness)  on  both  undergraduate  subjects  and 
experienced  personnel  selection  managers.  The  results 
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showed  that  the  undergraduates  rather  consistently  used  all 
three  irrelevant  dimensions  in  their  job  selection 
judgments.  In  contrast,  the  experienced  personnel  relied 
primarily  on  job-relevant  information,  with  less  use  of  sex 
information. 

The  important  fact  is  that  both  groups  did  make  some 
use  of  the  irrelevant  information.  Nagy's  demonstration  of 
such  usage  is  quite  profound.  The  irrelevant  information 
used  in  her  study  is  defined  to  be  irrelevant  both  by 
company  policy,  and  by  law.  Thus,  given  the  evidence  from 
Nagy's  study,  the  topic  of  personnel  selection  provides  a 
meaningful  research  area  in  which  to  investigate  the  effects 
of  training  on  the  use  of  irrelevant  information.  It  is 
also  an  area  in  which  subjects  are  readily  accessible  and  in 
which  research  is  straightforward. 

Purpose  and  Rationale  af  ths.  Present  study 

Motivated  by  the  success  of  the  training  procedures 
used  in  earlier  research  on  agricultural  judges  (Shanteau  & 
Gaeth,  1981)  ,  we  set  out  to  test  them  in  the  area  of 
personnel  selection.  Our  goals  in  doing  this  were 
threefold:  First,  we  felt  that  it  would  be  useful  to  adapt 
the  training  procedures  from  the  original  soil  study  and 
apply  them  to  personnel  selection.  If  any  generalizable 
training  procedures  are  ever  to  be  developed,  such 
adaptation  must  be  achieved. 

Second,  we  wanted  to  separate  the  effects  of  lecture 
and  the  interactive  training.  Because  of  the  limited 
availability  of  soil  scientists,  such  a  comparison  would  be 
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impractical.  However,  Nagy  demonstrated  that  students  could 
be  meaningfully  used  as  participants  in  a  personnel  study. 
Therefore,  a  large  subject  pool  -would  be  available  to 
compare  the  training  procedures. 

Finally,  we  were  motivated  by  the  importance  of  the 
area  of  personnel  selection.  It  would  be  of  no  small 
consequence  to  develop  a  usable  training  procedure-  which 
reduced  the  use  of  irrelevant  information  in  job  hiring. 
Any  such  procedure  would  be  likely  not  only  to  help 
businesses  comply  with  the  law,  but  also  to  increase  the 
effectiveness  of  the  personnel  selection  process. 

To  reach  the  point  where  the  training  procedures  could 
be  evaluated,  it  was  first  necessary  to  reproduce  the 
stimuli  used  in  Nagy's  research.  Fortunately,  we  were  able 
to  obtain  copies  of  the  job  application  forms  used  by  Nagy.1 
Using  these,  we  were  able  to  create  a  set  of  similar,  but 
improved  stimuli. 

The  basic  research  strategy  was  to  use  these  stimuli  in 
a  (1)  pre-test,  (2)  treatment,  (3)  post-test  design.  The 
two  training  procedures  then  served  as  the  treatments  to  be 
evaluated. 

Reservations 

It  is  necessary  to  keep  in  mind  the  tentative  nature  of 
the  initial  research  presented  here.  Although  the  details 


^The  authors  wish  to  thank  Geraldine  Nagy  for  providing  a 
complete  set  of  her  stimuli,  along  with  instructions. 
Without  this  material,  this  research  would  have  been 
impossible. 
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of  the  experimental  design  and  training  procedures  have  been 
developed,  the  analyses  are  based  on  only  a  few  subjects. 
In  addition,  the  instructions  were  slightly  adjusted  while 
this  initial  research  was  in  progress. 

Two  positive  points  should  be  noted.  Because  we 
followed  Nagy's  (1981)  stimuli  and  design,  our  results  can 
be  directly  compared  to  her  findings.  This  may  be  done 
independently  of  the  training  procedures  and  represents  a 
first  and  necessary  step  in  the  validation  of  our 
experimental  design.  Second,  the  revised  training 
procedures  are  clearly  superior  to  those  used  by  Shanteau 
and  Gaeth  (1981) .  Many  of  the  earlier  steps  have  been 
combined  and  made  more  efficient.  Moreover,  the  increased 
understanding  of  the  structure  underlying  the  training 
approach  should  now  make  further  generalization  to  other  new 
areas  much  easier. 

METHOD 

Subjects.  Twelve  subjects  were  recruited  through  a 
bulletin  board  advertisement  at  Kansas  State  University. 
They  were  each  paid  $3.00  per  hour  for  their  participation. 
There  were  six  males  and  six  females;  the  group  had  a  median 
age  of  22  years. 

The  two  training  procedures  were  being  developed  as 
this  initial  research  was  in  progress.  Thus,  the  training 
procedure  given  a  subject  was  chosen  more  by  what  was  ready 
for  testing  rather  than  by  a  pre-determined  sequence.  Pour 
subje  .  re  j.ved  only  the  pre-evaluation.  Five  subjects 
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were  given  the  pre-evaluation,  the  lecture  training,  and  the 
post-evaluation.  Two  subjects  received  the  pre-evaluation, 
the  interactive  training,  and  the  post-evaluation.  Finally, 
because  of  an  experimenter  error,  one  of  the  subjects  was 
given  an  incorrect  version  of  the  pre-test,  then  the  correct 
lecture  training,  and  the  correct  post-evaluation.  Because 
of  the  pilot  nature  of  this  study,  all  of  the  data  was 
retained  and  analyzed. 


Evaluation  Task.  The  pre-  and  post-evaluation 

2 

procedure  was  identical  to  the  one  used  by  Nagy.  The 
evaluation  task  required  the  participants  to  judge  the 
hiring  desirability  of  a  number  of  hypothetical  job 
applicants.  Each  was  described  by  a  job  application  form 
which  contained  a  written  information  sheet  and  a 
photograph.  An  example  of  one  of  the  job  application  forms 
is  given  in  Figure  1.  The  job  applicant  form  was  modeled 
after  one  actually  used  at  the  Weyerheauser  Corporation  in 
Washington  state. 

In  addition  to  various  filler  information  (name, 
address,  medical  record,  etc.),  used  to  make  the  form  more 
realistic,  each  form  contained  five  dimensions  of  critical 
information:  sex,  age,  attractiveness,  recommendations,  and 
related  job  experience.  In  the  set  of  stimuli,  each  of 
these  factors  was  varied  across  two  levels  as  follows: 


Sex — male. 


female; 


Age — old, 


young; 


Physical 


2The  reader  should  refer  to  Nagy  (1981)  for  more  details  as 
to  her  procedures.  Only  a  summary  will  be  given  here. 


APPLICANT  INFORMATION  FORM 


WEYERHAEUSER  CORPORATION 


for  position  :  Computer  programmer  /anal  vi  st 


Name :  Jane  Armstrong 


Address : 


\ 


Telephom  No. : 


Social  Security  No.: 


Age:  45 


Sex :  Female 


Position  desired:  Computer  programmer/analvist 


Is  applicant  willing  to  relocate?  yes 


Is  applicant  willing  to  work  evenings, 
weekends,  or  holidays  when  necessary?  vos 


Education:  Education  includes:  A -year,  r.ollpge 
degree  in  computer  programming /rnmpni-pr-  gn’cnro 


Eusiness  Experience :  Applicant  has  had 

approximately  8 _  years  of  relevant 

work  experience. 


Recommendations :  Recommendations  from  past 
employer (s)  or  supervisor (s)  were 
generally:  below  average _ 


Other :  Medical  history  and  examination 

results:  _ normal _ ___ 


DO  240 


Applicant  # 


Figure  1.  Typical  job  application  form 


[Pll  Redacted] 


A 


* 
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attractiveness — high,  low;  Recommendations-- above  average, 

below  average;  Experience — 1  year,  8  years.  The  photographs 

were  selected  by  Nagy  to  reflect  the  specific  levels  of  sex, 

3 

age  and  physical  attractiveness.  Thus,  a  total  of  eight 
photos  was  used  in  the  design. 

The  five  dimensions  of  information  discussed  above 
generated  a  2^  factorial.  In  addition  to  these  32 
experimental  stimuli,  8  fillers  were  added  to  disguise  the 
experimental  design.  The  stimuli  were  presented  in  a 
pseudorandom  order  under  the  stipulation  that  no  two 
application  forms  containing  the  same  photograph  would 
appear  in  sequence.  The  set  of  stimuli  were  repeated  once. 

The  desirability  judgments  were  recorded  by  the  subject 
on  a  100  mm.  line  in  a  response  booklet.  After  the  task  was 
explained,  several  practice  stimuli  were  presented.  Then 
the  subject  evaluated  each  of  the  application  forms;  this 
required  about  an  hour.  The  evaluation  procedure  was  run 
singly  or  in  groups  of  two,  while  the  training  procedures 
where  conducted  individually.  The  same  procedure  was  used 
for  both  the  pre-  and  post-evaluation  sessions. 

Lecture  Training.  The  lecture  training  was  developed 
with  a  structure  similar  to  the  one  used  in  our  previous 
agricultural  study  (Gaeth  &  Shanteau,  1981)  .  Basically, 
there  were  three  parts:  First,  a  definition  of  the 

irrelevant  factor  was  given  and  discussed.  Second,  evidence 

^These  photographs  were  pre-tested  by  Nagy  for 
attractiveness.  Our  results  supported  her  selection 
criteria. 
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was  presented  which  showed  that  a  particular  irrelevant 
factor  can  cause  errors  in  the  judgments  of  even  experienced 
decision  makers.  Third,  a  series  of  verbal  suggestions  were 
presented  which  were  designed  to  help  reduce  the  influence 
of  the  irrelevant  information.  During  training,  subjects 
received  instructions  only  on  how  to  avoid  the  biasing 
influence  of  sex  and  age.  No  mention  was  made  of 
attractivness;  this  was  purposely  done  so  that  the 
generality  (to  other  untrained  dimensions)  of  the  training 
procedures  could  be  evaluated. 

Although  the  complete  transcript  for  the  lecture 
training  is  given  in  Appendix  A,  it  may  be  helpful  to  give  a 
brief  summary  of  the  specific  content  of  the  three  parts. 
Part  one  described  the  legal  definition  of  bias  based  on  age 
and  sex.  Part  two  discussed  the  various  ways  these  biases 
could  influence  hiring  decisions;  also,  evidence  was 
graphically  presented  showing  that  experienced  personnel 
managers  suffered  from  such  biases.  Finally,  part  three 
contained  six  suggestions  designed  to  reduce  the  influence 
of  the  biasing  information.  Figure  2  provides  a  list  of 
these  suggestions.  It  is  important  to  note  that  at  no  time 
was  the  notion  of  irrelevance  mentioned;  these  issues  were 
always  referred  to  as  "personal  biases." 

Interactive  Training.  The  interactive  training 
involved  actual  practice  of  the  suggestions  presented  in  the 
lecture  training.  It  began  with  a  brief  statement  about  the 


LIST  OF  SIX  SUGGESTIONS 
TO  HELP  YOU  DEAL 
WITH  BIASING  INFORMATION 


1,  Evaluate  the  applicant  as  you  would  without 

CORRECTING  FOR  BIASES, 


2.  Determine  what  your  own  bias  is  for  age. 


3.  Determine  what  your  general  bias  is  for  sex. 


4.  Separate  the  biasing  information  of  sex  and 

AGE  FROM  THE  USEFUL  INFORMATION. 


5.  Correct  for  the  individual  biases  for  age  and  sex 


6.  Carefully  combine  the  two  biases  and  evaluate  the 
applicant  correcting  for  both  biases. 


Figure  2.  Suggestions  used  in  the  lecture  training. 
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irrelevant  dimensions  and  launched  immediately  into  active 
practice  of  the  suggestions.  Thus,  overlap  with  the  lecture 
training  was  minimal,  just  as  in  the  original  soil  study. 

The  interactive  training  was  divided  into  four  parts. 
The  procedure  is  summarized  below;  the  complete  transcripts 
are  available  in  Appendix  B.  In  the  first  part,  the  subject 
was  given  an  application  form  to  evaluate;  this  form  had 
been  judged  earlier  in  the  pre-evaluation  session  as  a 
filler.  The  only  difference  between  this  one  and  the 
original  was  a  change  in  the  irrelevant  information  (both 
sex  and  age) .  The  new  judgment  was  compared  to  the  old  one 
in  as  unfavorable  a  fashion  as  possible  (the  most  dissimilar 
of  the  two  judgments  in  the  pre-evaluation  session  was  used 
for  comparison) .  It  was  stressed  that  any  changes  in 
judgment  from  the  original  must  be  due  to  information  which 
caused  a  bias.  This  was  used  to  motivate  the  subject,  and 
show  them  that  they  personally  were  biased  by  this 
information. 

Next,  a  series  of  practice  stimuli  were  used  to  help 
the  subject  learn  to  deal  with  the  biasing  information  of 
age  and  sex.  This  was  accomplished  similarly  for  both 
factors  by  having  the  subjects  evaluate  applications  which 
had  successively  increasing  levels  of  bias.  For  example,  in 
the  case  of  the  age  factor,  the  subjects  saw  forms  with  no 
listed  age,  an  age  of  25,  an  age  of  55,  and  an  age  of  67. 
It  was  stressed  that  this  should  produce  no  changes  in  their 
judgments.  In  the  third  part  of  the  interactive  training 
the  same  procedure  was  used  for  the  sex  dimension. 
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Finally,  the  judge  was  given  another  application  form 
and  asked  to  make  a  judgment.  This  form  had  also  been  shown 
before  as  a  filler/  but  with  different  irrelevant 
information.  The  judgment  was  compared  to  the  pre¬ 
evaluation  judgments  in  as  favorable  a  way  as  possible  to 
show  the  subject  that  he/she  was  making  progress. 

RESULTS 

Use  Relevant  and  Irrelevant  Information 

The  first  goal  of  this  research  was  to  replicate  the 

previous  results  of  (Nagy,  1981)  which  showed  an  influence 

due  to  the  irrelevant  information  on  the  application  form. 

To  this  end,  the  responses  were  analyzed  using  a  25ANOVA  on 

each  individual  subject.  A  group  ANOVA  was  not  performed 

here  because,  as  pointed  out  by  Nagy,  there  are  likely  to  be 

considerable  differences  in  the  way  individual  subjects  use 

4 

the  information.  Such  differences  may  be  obscured  in  the 
group  analyses. 

The  results  of  the  individual  pre-evaluation  ANOVAs  are 
given  in  Table  1.  Looking  at  the  upper  half  of  Table  1  it 
is  immediately  obvious  that  every  subject  used  at  least  one 
of  the  two  relevant  dimensions  of  information  (either 
experience  or  recommendation)  as  reflected  in  a  main  effect. 
These  findings  are  strikingly  similar  to  Nagy's.  For  her 
subjects,-  100%  used  recommendations,  70%  used  the  experience 


4 

This  is  especially  true  for  the  irrelevant  information. 
For  example,  one  might  expect  an  interaction  between  sex  of 
the  subject  and  the  sex  of  the  applicant.  Similar 
situations  can  also  occur  with  age  and  attractiveness. 
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recommendation,  91%  used  experience,  and  36%  used  the 
interaction. 

Inspection  of  the  lower  half  of  Table  1  also  shows  that 
every  subject  used  some  form  of  irrelevant  information.  Six 
of  the  11  used  irrelevant  information  as  shown  in  main 
effects;  all  used  an  irrelevant  dimension  as  reflected  in 
the  higher  order  interactions.  Thus,  these  individual- 
subject  ANOVAs  clearly  confirm  the  l  elief  that  subjects  use 
irrelevant  information  as  well  as  relevant  information,  in 
making  personnel  selection  judgments. 

Efficacy  £f  the  Training  Procedures 

There  are  a  number  of  analyses  which  may  be  used  to 
evaluate  the  effectiveness  of  the  training  procedures.  The 
goal  for  this  report,  however,  is  not  to  present  an 
exhaustive  series  of  analyses.  Rather,  only  the  findings  of 
most  immediate  interest  will  be  presented. 

The  effectiveness  of  the  two  training  procedures  was 
tested  first  by  comparing  the  use  of  irrelevant  information 
in  the  pre-evaluation  (before  training)  with  its  use  in  the 
post-evaluation  (after  training) .  This  was  done  by 
performing  ANOVAs  on  the  post-evaluation  data  which  were 
identical  in  form  to  the  ones  described  above.  The  results 
are  shown  in  Table  2. 

In  all  subjects  but  #9,  a  general  shift  was  seen 
towards  less  use  of  irrelevant  information  after  training. 
There  is  both  a  decrease  in  the  number  of  irrelevant  effects 
and  an  increase  in  the  number  of  relevant  effects  that  are 
significant.  For  instance,  only  two  subjects  had 
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Table  2 

Post  training  Use  of  Information  as  Reflected  in 
Significant  Effects 

Subject 


S3 

S5 

S8 

S9 

S10 

Sll 

S6 

S7 

Training1 

Lect 

Lect 

Lect 

Lect 

Lect 

Lect 

Int. 

Int. 

Relevant 

Exp 

*2 

* 

* 

* 

* 

* 

Rec 

* 

* 

* 

* 

* 

* 

* 

E  X  R 

* 

* 

* 

* 

* 

Irrelevant 

Sex 

* 

* 

Att 

* 

Age 

* 

Higher 

Inter. 

4 

0 

1 

3 

0 

1 

1 

0 

*  Lecture  Training  Int.  »  Interactive  Training 
Icates  significance  at  £<.05. 
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significant  main  effects,  and  six  out  of  eight  subjects  had 
zero  or  one  irrelevant  interaction  effects. 

Additional  Analyses.  Based  on  her  results,  Nagy 
proposed  an  interesting  relationship  between  the  irrelevant 
information  and  level  of  relevant  information.  She 
speculated  that  when  the  level  of  the  relevant  factors  was 
high,  i.e.,  both  high  experience  and  high  recommendations, 
then  the  irrelevant  information  was  more  likely  to  be  used. 
This  effect  may  thought  of  as  a  "tie-breaker"  for  those 
applicants  who  are  otherwise  equally  qualified  on  all  the 
relevant  information.  To  further  investigate  the  role  of 
irrelevance,  this  relationship  was  pursued. 

To  understand  the  point  of  the  next  analysis  it  is 
necessary  to  restate  the  original  goal  for  the  training 
procedures:  After  training,  the  judgments  of  all  stimuli 
which  differ  on  only  irrelevant  dimensions  should  be  equal 
(except  for  error) .  To  test  this  prediction,  the  data  have 
been  plotted  in  Figure  3  across  the  8  levels  of  irrelevance 
for  the  high  experience,  high  recommendation  levels  of 
relevant  information.  This  was  done  both  for  lecture 
training  and  interactive  training.  Using  these  plots,  the 
personnel  selection  judgments  are  displayed  as  a  function  of 
the  irrelevant  information.  If  the  irrelevant  information 
has  no  influence,  then  the  plotted  points  should  be  flat, 
i.e.,  no  difference.  This  is  clearly  not  the  case  for  the 
pre-evaluation  (solid  lines)  judgments. 

To  check  for  the  effectiveness  of  the  training 
procedures,  the  post-evalutaion  data  was  also  plotted 


Lecture  Training  (N  =  5) 
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Interactive  Training  (n  =  2) 
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unart.  attrac.  attrac.  unatt.  unatt-  attrac- 

young  old  young  old  young  old 


Figure  3 .  Effectiveness  of  training  in  reducing  influence  of 

irrelevant  information  for  the  high  experience,  high 
recommendation  stimuli. 
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(dotted  lines)  in  Figure  3.  As  can  be  seen  by  the  flatness 
of  the  line,  the  effect  of  the  irrelevant  information  was 
greatly  reduced  by  the  training.  Although  group  data 
appears  in  Figure  3,  individual  subjects'  plots  also  confirm 
this  result. 

Although  a  strong  case  can  be  made  for  the  impact  of 
training,  there  is  another  check.  The  relevant  factors  may 
be  thought  of  as  occuring  at  4  different  levels  (2  levels  of 
experience  X  2  levels  of  recommendations) .  For  any  fixed 
level  of  relevant  information,  the  irrelevant  information 
varied  across  8  levels  (2  levels  of  sex  X  2  levels  of  age  X 
2  levels  of  physical  attractiveness) .  With  this  in  mind, 
the  prediction  that  the  training  reduces  the  influence  of 
the  irrelevant  information  is  reflected  in  a  parallel 
prediction  about  variances.  That  is,  when  a  relevant  factor 
level  is  held  constant,  the  judgments  should  show  decreased 
variance  when  the  irrelevant  factors  have  no  influence.  I  i 
fact,  if  the  training  worked  perfectly,  the  systemmatic 
variance  should  be  reduced  to  zero. 

To  test  this,  the  variance  over  the  judgments  (average 
of  the  two  replicates)  was  computed  across  the  8  levels  of 
irrelevant  information.  These  group  results  are  presented 
in  Table  3.  As  can  be  seen,  almost  all  of  the  variances 
decreased  after  training  (except  in  the  high-experience,  low 
recommendation  instance) .  Thus,  the  positive  impact  of  the 
training  procedures  can  be  seen  to  occur  throughout  all  of 
the  data. 
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Table  3 

Comparison  of  Pre-  and  Post-  Training  Variance  Caused  by  the 

Irrelevant  Information 


Relevant  Information 


High  Exp  High  Exp  Low  Exp  Low  Exp 

High  Rec  Low  Rec  High  Rec  Low  Rec 


Lecture  Training 

Pre-  142.32  115.56  83.72  118.59 

Post-  57.55  120.34  41.73  91.01 


Inter.  Training 
Pre- 
post- 


141.37 

48.44 


73.10 

278.22 


46.51 

30.91 


48.44 

35.40 
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DISCOS  SION 


implications  Results 

Based  on  both  the  original  study  by  Nagy  (1981)  and  the 
results  of  the  work  in  progress  discussed  here,  it  has  been 
shown  that  the  irrelevant  information  of  sex,  age  and 
attractiveness  inappropriately  influence  the  hiring 
judgments  of  subjects.  However,  this  influence  can  be 
reduced  by  training.  Moreover,  both  the  lecture  and 
interactive  training  procedures  proved  to  be  effective. 
These  two  issues  concerning  the  use  of  irrelevance  and  the 
effects  of  training  are  worth  further  discussion. 

First,  the  ability  to  replicate  the  results  of  Nagy's 
(1981)  study  was  essential  for  our  purposes.  In  fact,  the 
similarities  between  our  results  and  Nagy's  are  striking. 
Comparable  results  were  found  in  the  usage  of  both  relevant 
and  irrelevant  information.  More  importantly,  all  of  the 
subjects  made  some  initial  use  of  irrelevant  information. 

Second,  the  training  procedures  as  developed  for  the 
hiring  selection  problem  were  encouragingly  successful. 
Despite  the  small  number  of  subjects,  the  evidence  points  to 
a  consistent  reduction  in  the  influence  of  irrelevance  after 
the  lecture  training.  While  there  were  too  few  subjects  to 
speak  in  a  meaningful  way  about  interactive  training,  the 
results  also  indicate  a  positive  trend. 

One  comment:  We  were  able  to  adapt  the  basic  material 
from  our  original  agricultural  training  procedures  and 
restructure  it  so  that  it  could  be  applied  to  hiring 
decisions.  While  this  turned  out  to  be  considerably  more 
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difficult  than  initially  thought,  it  was  successful. 
However,  the  true  test  of  the  value  of  this  effort  will 
occur  when  we  attempt  to  generalize  the  procedures  again. 
All  indications  are  that  it  will  be  considerably  easier  the 
next  time. 

Research  la  PtJMISSS 

Prom  the  foundation  created  by  this  study,  we  are 
presently  carrying  out  a  large  scale  experiment.  Our  goals 
are  to  evaluate  the  lecture  and  interactive  training 
procedures  independently.  In  this  large  experiment,  there 
are  four  groups  of  twenty  subjects,  each  exposed  to  a 
different  training  condition. 

The  first  group  is  being  given  only  a  pre-evaluation 
and  a  post-evaluation.  This  represents  the  control  group, 
and  will  be  used  as  a  baseline  to  compare  the  other  training 
groups.  Both  the  second  and  third  groups  have  a  pre-  and 
post-evaluation.  However,  the  second  group  receives  only 
the  lecture  training  while  the  third  is  given  only  the 
interactive  training.  Finally,  the  fourth  group  receives 
both  the  lecture  training  and  the  interactive  training  in 
sequence,  with  an  pre-,  mid-,  and  post-evalution.  In  this 
way,  it  is  hoped  we  can  further  analyze  and  refine  the 
training  procedures. 
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APPENDIX  A 

Lecture  Training  Protocol 
tlfig&ice.  Training  £su.  Biases  1 

For  several  years  now  a  research  group  here  at  KSU  has 
been  studying  how  personnel  managers  make  hiring 
recommendations.  I  have  been  interested  in  this  research 
because  hiring  recommendations  are  so  important  to  a 
company . 

Each  hiring  decision  involves  a  great  deal  of  money 
and  a  considerable  time  committment  both  for  the  applicant 
and  for  the  company  doing  the  hiring.  In  addition,  usually 
only  one  applicant  can  be  selected.  But  then,  if  a  wrong 
recommendation  is  made,  it  may  never  be  possible  to  know 
which  applicant  was  really  the  best.  For  these  reasons, 
personnel  selection  must  be  made  as  accurately  and 
carefully  as  possible. 

We  have  found  that  the  success  rate  of  the  job 
recommendations  is  generally  quite  high.  By  this,  I  mean 
that  frequently  the  most  qualified  applicant  is  correcly 
selected.  However,  when  a  wrong  recommendation  is  made, 
and  the  best  applicant  is  not  selected,  this  often  seems  to 
be  due  to  biases  caused  by  such  information  as  age  and  sex. 
As  you  probably  know,  age  and  sex  legally  should  not 
influence  the  hiring  recommendation  in  any  way. 

What  I  would  like  to  talk  to  you  about  today  are  some 
of  the  methods  which  might  be  used  to  prevent  these  illegal 
biases  from  influencing  your  own  personal  hiring 
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recommendations.  I  am  specifically  interested  in 
discussing  how  you  might  go  about  evaluating  an  applicant's 
job  potential  without  being  influenced  by  the  applicant's 
age  or  sex. 

Almost  of  us  have  biases  of  one  sort  or  another.  You 
should  realize  that  these  personal  biases  do  exist  and  that 
they  are  very  hard  to  permanently  change.  We  will  not  be 
concerned  with  totally  remove  these  biases  here.  Instead, 
we  will  only  try  to  show  you  how  to  avoid  having  personal 
biases  influence  your  job  recommendations. 

As  I  said  earlier,  it  is  against  the  law  to  let  any 
biases  toward  age  or  sex  affect  hiring  decisions  or 
recommendations.  Remember,  the  point  is  that  we  all  tend 
to  have  biases  of  some  sort  or  another.  So,  it  is  likely 
that  both  age  and  sex  will  influence  you  in  some  way. 
However,  hopefully  from  this  discussion  you  can  learn  to 
control  for  the  influence  that  age  and  sex  have  on  your  job 
recommendations . 

In  the  remainder  of  this  session  we  will  go  over  three 
things.  First,  we  will  discuss  the  legal  definition  of 
bias  due  to  age  and  sex.  In  the  second  part,  we  will  look 
at  how  information  on  the  age  and  sex  of  the  applicant  may 
bias  the  judgment  of  even  highly  trained  personnel 
managers.  And  then  third.  I  will  give  you  some  ideas 
designed  to  help  you  personally  make  less  biased  hiring 
recommendations  for  job  applicants. 
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[ Definition  of  Bias  diie  ia  Age  end  see] 

Let  me  start  by  explaining  definition  of  bias  due  to 
age  and  sex.  I  have  looked  up,  and  would  like  to  present 
to  you,  some  laws  defining  the  use  of  age  and  sex  in  hiring 
recommendations.  We  will  consider  the  two  issues  of  age 
and  sex  separately. 

Federal  regulations  cover  age  discrimination  in  a  law 
called  Title  II.  This  law  was  passed  in  1975  and  says  that 
discrimination  on  the  basis  of  age,  as  part  of  any  hiring 
practice,  is  prohibited.  Here  we  are  speaking  of  all  jcb 
applicants  under  retirement  age.  The  government  felt  so 
strongly  about  this  issue  of  age  discrimination  that  the 
penalty  imposed  on  a  guilty  company  was  to  have  all  Federal 
aid  removed.  Also,  the  person  who  was  refused  the  job  is 
often  able  to  sue  the  employer  to  obtain  a  settlement  for  a 
considerable  amount  of  money. 

However,  even  after  this  law  was  in  effect,  age  was 
still  illegally  used  as  part  of  some  hiring  decisions. 
Because  of  this,  in  1979  President  Carter  issued  an 
executive  order  stating  that  all  businesses  had  to  provide 
additional  records  for  inspection  to  determine  if  there  had 
been  discrimination  based  on  age.  That  is,  the  government 
has  the  right  to  investigate  a  business  to  see  if  they  have 
discriminated  against  job  applicants  because  of  age.  The 
point  is  that  this  and  the  original  law  both  clearly  state 
that  age  is  not  to  be  used  as  part  of  any  hiring  decision. 

Essentially  the  same  type  of  laws  apply  for 

discrimination  due  to  sex.  A  similar  law,  called  Title 
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VII,  states  that  information  on  the  sex  of  a  the  applicant 
is  not  to  be  used  as  part  of  hiring  recommendation.  This 
law  passed  in  1964,  includes  all  aspects  of  hiring  such  as 
advertisement  for  the  job,  employment  selection,  employment 
requirements,  recruitment,  job  classification,  and  fringe 
benefits. 

I  am  sure  you  are  aware  the  same  law  states  that  women 
should  receive  equal  pay  for  equal  work.  Did  you  also  know 
that  this  even  extends  to  fringe  benefits?  For  example, 
pregnant  employees  must  be  offered  the  same  benefits  as  are 
offered  to  any  temporarily  disabled  employees. 

In  addition  to  leave  time,  an  employer  may  not  exclude 
from  employment,  any  employee  or  applicant  on  the  basis  of 
pregnancy.  Again,  violations  of  this  rule  will  result  in 
the  employer  losing  Federal  Aid.  It  could  also  result  in  a 
law  suit.  Therefore  the  law  clearly  states  that  sex  should 
have  no  influence  on  a  personnel  manager's  hiring 
recommendation. 

So,  you  have  seen  that  neither  the  age  of  an  applicant 
nor  his  or  her  sex  should  be  used  as  any  part  of  a  hiring 
recommendation.  Do  you  have  any  questions  about  what  I 
have  said?  [Answer  any  questions] . 
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f Demonstration  that  Aas,  and  SlXas.  Judqmentl 

Let  me  demonstrate  that  age  and  sex  do  influence 
hiring  recommendations. 

We  have  already  mentioned  that  each  of  us  has  personal 
biases.  This,  of  course,  includes  personnel  managers. 
Evidence  exists  that  some  very  experienced  personnel 
managers  may  allow  biases  to  influence  their  hiring 
recommendations.  Next,  we  will  discuss  some  evidence 
showing  that  age  and  sex  do  indeed  bias  the  judgments  of 
even  professional  personnel  managers. 

Let's  first  consider  bias  due  to  age.  We  want  to  be 
sure  that  you  understand  that  most  research  shows  that 
older  employees  are  just  as  effectve  as  younger  ones.  In 
fact,  although  some  physical  capabilities  may  decrease  with 
age,  these  small  losses  are  almost  always  made  up  for  by 
increased  experience. 

Evidence  for  age  bias  in  hiring  can  be  indirectly 
determined  by  considering  the  proportion  of  the  work  force, 
at  different  ages,  who  are  employed.  This  is  done  by 
combining  information  from  all  jobs  ac  oss  the  country. 
For  example,  we  can  consider  what  percentage  of  the 
available  25  year-olds  are  employed  — This  is,  in  fact, 
roughly  95% .  This  can  be  compared  to  the  percentage  of 
available  people  aged  61  who  are  employed.  What  do  you 
think  this  percentage  is?  [Get  response] .  It  is  actually 
somewhere  around  30%  or  less.  Because  we  are  speaking  of 
percentages,  if  there  were  no  bias  due  to  age,  then  these 
percentages  should  be  similar.  If  we  graph  the 
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relationship  between  age  and  percentage  hired  it  would  look 
like  this  [show  graph]. 

This  graph  demonstrates  the  percentage  of  employed 
persons  for  each  age  group.  Remember,  this  is  how  many  are 
employed  out  of  the  percentage  of  the  total  workers 
available.  The  lower  the  curve,  the  lower  the  percentage 
of  the  available  population  who  are  employed.  Notice  that 
the  curve  decreases  very  rapidly  after  about  the  age  of  50. 
This  indicates  that  starting  about  age  50,  a  considerable 
bias  exists.  Do  you  see  the  decrease  ?  [Point] .  If 
there  was  no  bias  due  to  age,  this  curve  should  extend  in  a 
perfectly  straight  line.  Do  you  understand  what  is  being 
shown  in  the  graph? 

The  sex  of  a  job  applicant  is  also  known  to  influence 
the  evaluation  by  some’  personnel  managers.  How  much  it 
influences  the  applicant,  and  whether  it  is  favorable  or 
unfavorable,  depends  to  a  great  extent  on  the  type  of  job. 
Let  me  try  and  explain  this  to  you. 

First  of  all,  many  jobs  may  be  classified  as  typically 
male  or  typically  female.  For  example,  a  nurse  might  be 
considered  to  be  a  typically  female  job,  while  a 
construction  worker  might  be  considered  to  be  a  male  job. 

In  the  case  of  a  typically  male  job,  it  is  very  likely 
that  females  will  be  discriminated  against  and  their 
applications  will  be  given  lower  recommendations.  That  is, 
for  the  job  of  a  construction  worker,  the  male  applicant  is 
likely  to  be  given  a  higher  rating  than  a  female  applicant 
even  if  they  are  equally  qualified. 
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This  situation  is  shown  in  the  first  panel  of  this 
graph  [point] .  As  you  can  see  ,  males  are  given  a  much 
higher  recommendation  for  the  male  oriented  job  than  are 
females.  [point]  Do  you  understand  what  the  graph  is 
showing? 

This  situation  is  reversed  in  the  case  of  a  typically 
female  job.  For  female  jobs,  often  the  personnel  manager 
will  be  biased  against  the  male  applicant.  This  situation 
is  again  shown  in  the  graph  [point] .  In  this  case  you  can 
see  that  for  female  jobs,  the  males  are  given  a  lower 
evaluation. 

It  is  also  possible  that  the  personnel  manager  may  be 
biased  totally  against  one  sex  or  the  other,  no  matter  what 
the  job.  In  this  case,  there  would  be  a  consistent  bias 
against  women.  Therefore,  no  matter  what  the  job,  the 
female  applicant  will  be  discriminated  against.  The 
reverse  could  also  be  true:  There  could  be  an  across-the- 
board  bias  against  men.  Bias  due  to  sex  is  present  even 
though  this  is  against  the  law. 
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Desianed  la  Reduce 


I  have  six  suggestions  which  you  may  find  personally 
useful  in  reducing  discrimination  in  your  evaluation  of  job 
applicants.  I  have  listed  them  here  on  this  chart  [point] 
which  we  will  follow  as  we  discuss  each  point. 

The  first  suggestion  is  easy.  You  should  evaluate  the 
applicant  as  you  would  normally.  That  is,  don't  try  to 
correct  for  your  biases,  just  do  what  comes  naturally.  The 
reason  to  do  this  is  to  give  you  a  starting  point  in  the 
evaluation  of  a  job  applicant. 

The  second  step  is  for  you  to  consider  the  general 
biases  you  may  have  toward  age.  For  example,  you  may  have 
a  personal  bias  for  applicants  above  or  below  a  certain  age 
group.  Do  you  think  you  have  an  age  bias?  what  is  it? 
[Get  response]  Good.  Do  you  think  this  age  bias  is  strong 
or  weak?  [get  response] . 

We  all  have  our  own  biases  and  as  long  as  they  don't 
affect  our  hiring  decisions,  we  are  not  concerned  with  them 
here.  However,  your  biases  should  not  be  allowed  to 
influence  your  hiring  recommendations.  For  instance,  if  a 
personnel  manager  believes  that  a  person  must  be  under  age 
45  to  be  a  good  worker,  then  this  is  discrimination  and  can 
not  be  allowed.  That  is  because  the  the  manager  would  then 
prefer  an  applicant  of  that  age  even  though  the  age 
information  should  make  no  difference. 

As  the  third  step,  consider  your  personal  sex  bias. 
In  the  case  of  the  sex  information,  you  may  also  prefer  an 
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individual  who  is  the  same  sex  as  you  are.  What  do  you 
think  your  personal  sex  bias  is?  [get  response] .  Is  it 
strong  or  weak?  [get  response] .  Here  again,  this  bias  may 
lead  you  to  inappropriately  favor  an  applicant  of  that  sex 
over  another  equally  qualified  applicant.  Remember,  the 
second  and  third  steps  ask  you  to  recognize  the  direction 
and  size  of  your  own  biases. 

The  fourth  step  is  for  you  to  separate  age  and  sex 
biases  from  the  rest  of  the  information  about  the 
applicant.  The  reason  for  making  this  separation  is  very 
important.  As  you  know  a  job  application  form  also 
contains  useful  information  which  you  should  use  as  part  of 
your  job  recommendation.  We  do  not  want  you  to  ignore  all 
of  the  pieces  of  information  about  the  applicant;  just  the 
inis tmat ion  which  miaht  bias  xaii. 

To  help  you  use  the  fourth  step  and  effectively 
separate  out  the  biasing  information  you  should  first  note 
the  sex  of  the  applicant.  Then  think  about  the  bias  it 
produces  in  you.  Next,  the  same  should  be  done  for  the  age 
of  the  applicant  and  your  corresponding  bias.  It  is 
important  that  you  decide  whether  you  think  the  sex  and  age 
of  the  applicant  biases  you  favorably,  or  unfavorably.  Of 
course,  this  will  depend  on  you  personally. 

As  the  fifth  step,  now  that  you  have  decided  what 
these  biases  are,  you  should  try  to  make  a  correction  for 
them.  You  should  keep  in  mind  and  make  every  effort  to 
prevent  the  age  or  the  sex  of  an  applicant  from  having  an 
effect  on  your  hiring  recommendation. 
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Finally,  for  the  sixth  and  last  step,  we  will  remind 
you  of  all  the  other  suggestions.  Up  to  this  point,  you 
have  evaluated  the  applicant  as  you  would  if  you  allowed 
the  biases  to  influence  your  hiring  recommendation.  Then 
you  evaluated  your  own  personal  biases  due  to  sex  and  age. 
Next  you  decided  on  the  size  and  direction  of  these 
biases,  and  have  made  an  effort  to  separate  the  biasing 
information  from  the  useful  information.  Finally,  you 
attempted  to  adjust  for  the  biases. 

Now,  for  the  sixth  and  last  step,  you  should  feel 
comfortable  that  you  are  ready  to  put  it  all  together  to 
determine  your  overall  bias  toward  the  applicant.  This  may 
be  harder  than  you  think  and  it  need  to  be  done  carefully. 
There  are  many  different  ways  that  two  separate  biases  may 
combine  to  form  one  overall  bias.  For  example,  one  bias 
may  be  favorable,  while  the  other  is  unfavorable  and 
therefore,  they  may  somewhat  cancel  each  other  out.  Or  the 
biases  may  both  be  in  the  same  direction,  say  unfavorably, 
and  add  up  to  even  larger  unfavorable  bias. 

You  must  carefully  evaluate  all  your  biases  together 
and  estimate  an  overall  bias.  Once  you  have  done  this,  you 
are  in  a  position  to  reevaluate  the  applicant  after 
correcting  for  whatever  biases  are  present.  Remember,  as  a 
first  step  you  evaluated  the  applicant  as  you  would 
normally  without  correcting  for  your  biases.  Now  you  can 
make  a  final  different  and  unbiased  recommendation.  In 
this  way  I  think  you  can  learn  how  to  handle  your  biases  in 
a  constructive  manner. 
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List  of  Six  Suggestions  to  Help  you 
Deal  with  Biasing  Information 

Evaluate  the  applicant  as  you  would  without 
correcting  for  biases. 

Determine  what  your  own  bias  is  for  age. 

Determine  what  your  general  bias  is  for  sex. 

Separate  the  biasing  information  of  sex  and  age  from  the 
useful  information. 

Correct  of  the  individual  biases  for  age  and  sex. 


Carefully  combine  the  two  biases  and  evaluate  the 
applicant  correcting  for  both  biases. 
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APPENDIX  B 


Interactive  Training  Protocol 
Interactive  Bias  Training 

For  several  years  now  a  research  group  here  at  KSU  has  been 
studying  how  personnel  managers  make  hiring  recommendations. 
I  have  been  interested  in  this  research  because  hiring 
recommendations  are  so  important  to  everyone  involved. 

Each  hiring  decision  involves  a  great  deal  of  money  and  a 
considerable  time  committment  both  for  the  applicant  and  for 
the  company  doing  the  hiring.  In  addition,  usually  only  one 
applicant  can  be  selected.  But  then,  if  a  wrong 
recommendation  is  made,  it  may  never  be  possible  to  know 
which  applicant  was  really  the  best.  For  these  reasons, 
personnel  selection  must  be  made  as  accurately  and  carefully 
as  possible. 

We  have  found  that  the  success  rate  of  the  job 
recommendations  is  generally  quite  high.  By  this,  I  mean 
that  frequently  the  most  qualified  applicant  is  correctly 
selected.  However,  when  a  wrong  recommendation  is  made,  and 
the  best  applicant  is  not  selected,  this  often  seems  to  be 
due  to  biases  caused  by  information  such  as  age  and  sex.  As 
you  probably  know,  age  and  sex  legally  should  not  influence 
the  hiring  recommendation  in  any  way. 

What  I  would  like  to  talk  to  you  about  today  are  some  of 
the  methods  which  might  be  used  to  prevent  age  and  sex 
biases  from  influencing  your  own  personal  hiring 
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recommendations.  I  am  specifically  interested  in  discussing 
how  you  might  go  about  evaluating  an  applicant's  job 
potential  without  being  influenced  by  that  applicant's  age 
or  sex.  The  procedure  we  will  be  using  involves  going 
through  a  series  of  steps  designed  to  aid  you  in  evaluating 
applications  without  any  bias. 

We  will  begin  by  having  you  evaluate  this  application  [old, 
male].  Could  you  please  read  it  over  and  then  respond  using 
the  same  slash  mark  procedure  as  you  did  in  the  earlier 
session  [record  response] .  Good. 

Okay,  this  same  application  form  with  essentially  the  same 
information  was  evaluated  by  you  in  the  earlier  session. 
The  only  difference  between  this  one  you  see  now  and  the 
earlier  one  was  a  change  in  the  information  on  age  and  sex. 
As  I  said  earlier,  legally  neither  age  nor  sex  information 
should  influence  your  evaluation.  To  find  out  if  there  is 
any  change,  I  looked  up  what  you  said  before. 

In  the  other  session  you  evaluated  this  same  basic 
applicant  [show  the  old  application  form]  with  a  slash  mark 
placed  here  [demonstrate-use  the  most  disparate  value] ,  but 
today  you  gave  an  evaluation  of  [point  to  the  new  slash 
mark] .  Notice  that  your  recommendations  differ 
considerably.  The  [lower,  higher]  response  you  gave  today 
could  only  have  been  due  to  the  bias  caused  by  information 
of  age  and  sex.  This  is  because  the  other  information  on 
the  application  form  was  identical.  [Point  and  show  them 
this]  .  Do  you  understand  this? 

As  I  have  pointed  put  these  two  evaluations  you  made  should 
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have  been  identical  because  the  information  of  age  and  sex 
should  not  have  influenced  you.  This  is  exactly  the  problem 
we  would  like  to  deal  with  in  the  remainder  of  this  session. 
I  will  give  you  a  series  of  suggestions  and  ideas  which  are 
aimed  at  helping  you  control  your  biases.  We  will  practice 
each  of  the  suggestions  on  other  applications. 

The  first  suggestion  is  easy.  Initially,  you  should 
evcluate  the  applicant  as  you  would  normally.  That  is, 
don't  try  to  correct  for  your  biases.  Just  do  what  comes 
naturally.  The  reason  to  do  this  is  to  give  you  a  starting 
point  in  the  evaluation  of  a  job  applicant. 

Each  of  us  has  biases  of  the  sort  we  are  considering  today. 
I  am  only  trying  to  help  you  discover  what  yours  are.  Next, 
we  will  talk  about  some  specific  things  you  can  do  to 
prevent  these  biases  from  affecting  your  applicant 
recommendations . 


We  will  look  at  the  problem  of 

bias 

caused 

by 

age 

information. 

At  this  point  we 

are 

interested 

in 

specifically 

dealing  with  the  influence 

of  age 

in 

you 

recommendations.  Consider  this  application  [no  photograph, 
no  sex,  no  age] .  Could  you  evaluate  it  please  [record 
response] .  Notice  that  the  information  on  age  is  not 
included,  but  all  of  the  critical  information  is  there. 
Good. 

Now  evaluate  this  application  [24-young  (circled  in  red) , 
no  sex,  no  photograph],  [record  response].  Good.  Notice 
this  time  the  age  of  the  applicant  is  circled  in  red.  In 
this  case,  do  you  think  that  the  age  biases  you  in  favor  or 
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attention  to  it.  It  is  important  to  consider  the  sex  of  the 
applicant  and  then  to  decide  whether  you  think  the  sex  may 
bias  you  personally.  You  should  decide  whether  you  think 
the  sex  bias  will  be  either  in  the  favor  or  against  this 
female  applicant.  In  this  instance  could  you  report  to  me 

how  you  think  the  bias  would  affect  your  evaluation  - 

that  is,  either  favorably  or  unfavorably  [record  response] . 

Go  ahead  now  and  give  your  recommendation  as  you  would  if 
you  allowed  the  bias  to  influence  you  [record  response] . 
Now  correct  for  the  bias  due  to  sex,  and  give  your 
evaluation  [record  response] .  Good.  You  are  learning  to 
compensate  for  the  bias  introduced  by  knowing  the  sex  of  the 
applicant. 

Now  look  at  this  application  [same  but  male,  (circled  in 
red)].  You  should  decide  whether  you  think  bias  due  to  sex 
will  be  in  favor  or  disfavor  of  the  applicant.  Report  to  me 
in  what  direction  you  think  the  bias  would  affect  your 
evaluation  in  this  case.  [record  response] .  Now  evaluate 
it  as  you  would  if  you  allowed  the  sex  bias  to  influence 
your  recommendation  [record  response] .  Now  correct  for  the 
bias  and  give  me  your  recommendation  again  [record 
response] .  Good.  Did  you  realize  that  all  three  of  these 
applications  were  the  same  except  for  differences  in  sex 
information?  [record  response] .  That  means  you  should  have 
given  them  exactly  the  same  recommendation. 

Now  consider  this  application.  [male,  young] .  Did  you 
note  the  age  and  sex  first  so  that  you  would  be  sure  to  be 
aware  of  them?  Did  the  age  bias  you  for  or  against  the 
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against  the  applicant  [record  response]?  How  would  you 
evaluate  the  applicant  if  you  allowed  the  bias  to  influence 
you  [record  response]?  Now  how  would  you  evaluate  it  if  you 
corrected  for  the  bias  [record  response]?  Good. 

Here  is  another  applicant  [45-old  (circled  in  red) ,  no  sex, 
no  photo] .  How  do  you  think  the  increased  age  would  bias 
you?  [record  response] .  How  would  you  rate  this  applicant 
without  correcting  for  the  age  bias?  [record  response] .  If 
you  corrected  for  it.  [record  response].  Good. 

Here  is  another  applicant  who  is  even  older  [66-old, 
(circled  in  red) ,  no  sex,  no  photograph] .  How  would  you 
evalute  it  after  you  have  corrected  for  the  bias  [record 
response] .  Good.  I  am  sure  by  this  time  you  realized  that 
all  of  these  applications  were  the  same  except  for 
differences  in  age.  These  differences  in  age  should  not 
have  influenced  your  recommendations  in  any  way,  right? 
That  has  been  what  we  have  been  practicing. 

Now  we  will  deal  with  the  bias  due  to  sex  information. 
Here  is  a  new  application  form  [no  photograph,  no  age,  no 
sex]  for  you  to  consider  and  evaluate.  Notice  that  the 
information  on  sex  is  not  included,  but  all  of  the  critical 
information  is  there.  Could  you  evaluate  the  application? 
[record  response] .  Good. 

Now  consider  this  application  [female,  but  the  same 
application]  Then  you  will  notice  that  this  time  the 
information  on  sex  is  there  and  circled  in  red,  but  all 
other  critical  information  is  the  same  This  was  done  so  that 
the  sex  of  the  applicant  would  stand  out  and  you  would  pay 
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applicant?  [record  response] .  Old  the  sex  bias  you  for  or 
against  the  applicant?  [record  response] .  How  would  you 
have  evaluated  the  applicant  without  correcting  for  the  bias 
[record  response] .  Good.  How  would  you  have  evaluated  the 
applicant  after  correcting  for  the  bias?  [record  Response] . 
I  am  trying  to  help  you  get  used  to  separating  the 
information  on  the  job  which  may  bias  you  from  the 
information  which  you  should  appropriately  use  to  make  your 
recommendation . 

The  next  step  is  designed  to  help  you  control  for  the 
problem  of  age  and  sex  biases  together.  This  may  be  much 
more  difficult.  One  reason  for  this  increased  difficulty  is 
that  sometimes  the  biases  and  cancel  each  other  out. 
However,  you  have  no  guarantee  that  this  will  happen  and 
should  not  count  on  it.  For  example,  if  both  age  and  sex 
bias  you  against  the  applicant  then  when  these  are  combined 
the  total  bias  may  be  quite  extreme.  For  these  reasons,  it 
is  important  to  consider  each  one  of  the  biasing  pieces  of 
information  separately  here  in  order  to  ensure  that  you  have 
made  the  most  unbiased  judgment  possible. 

Consider  this  application,  [female,  old].  I  would  like 
you  to  evaluate  this  applicant  correcting  for  age  and  sex 
bias.  The  only  difference  is  that  I  will  ask  you  to  report 
the  process  to  me.  First,  evaluate  the  applicant  on  your 
own  without  correcting  for  the  bias  and  report  it  to  me 
[record  response] .  Next,  determine  the  sex  and  age  of  the 
applicant  and  report  them  to  me  [record  response].  Great. 
Next,  decide  whether  each  piece  of  information  would  tend  to 
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bias  you  favorably  or  unfavorably.  Report  the  age  bias 

[record  response].  Report  the  sex  bias  to  me  [record 
response] .  Remember  to  combine  the  two  biases  carefully. 
Report  this  combination  of  biases  to  me  [record  response] . 
Now  correct  for  it  and  report  it  to  me  [record  response] . 

Finally,  evaluate  this  application  [old,  female]  using  all 
of  the  techniques  we  have  learned  today.  Good.  Actually, 
you  saw  this  application  before  [young,  male]  ;  nd  the  only 
difference  was  in  the  sex  and  age.  Last  time  it  was  a  young 

male  and  you  evaluted  him  as  _  [use  the  nearest  value] . 

This  time  you  said  _  for  the  older  female.  Notice, 

this  time  your  evaluations  are  much  closer,  that  is  very 
good. 

We  would  like  you  to  try  and  use  these  suggestions  when  you 
evaluate  applicants  in  the  final  session  [schedule  them  for 
it] . 


